Image processing apparatus, image processing method, image processing program recording medium, color adjustment method, color adjustment device, and color adjustment control program recording medium

ABSTRACT

An image processing apparatus inputs photographic image data containing dot-matrix pixels and converts the input pixels according to predetermined correspondence relationship information, so as to produce processed image data. A correspondence relationship holding unit of the apparatus holds correspondence relationship information, including a plurality of correspondence relationships, for the conversion, and applicable object information. The applicable object information includes, for each correspondence relationship, a respective condition. A correspondence relationship judging unit judges, for each of the input pixels, indicating whether a given pixel meets the condition of any of the correspondence relationships. An image data converting unit performs a conversion operation on the pixels of the input image data to produce processed image data. For each pixel, however, the conversion operation is performed using only the correspondence relationships indicated by the judgments of the correspondence relationship judging unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Continuation Application of U.S. application Ser. No.10/270,485 filed Oct. 16, 2002, which is a Continuation of applicationSer. No. 09/097,828 filed Jun. 16, 1998, issued as U.S. Pat. No.6,535,301 on Mar. 18, 2003. The entire disclosure of the priorapplications are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, an imageprocessing method and an image processing program recording medium forinputting digital photographic image data containing dot-matrix pixelsand converting each pixel of the image data according to predeterminedcorrespondence relationship information, and it is also concerned with acolor adjustment method, a color adjustment device and a coloradjustment control program recording medium for carrying out optimumcolor adjustment.

2. Description of Related Art

There are a variety of conventional image processing techniques forprocessing digital photographic image data, more specifically such imageprocessing techniques for contrast enhancement, chromaticity correction,and brightness correction, for example. Each pixel of image data isconverted according to predetermined correspondence relationshipinformation in these conventional image processing techniques. In anexample of chromaticity correction, a color conversion table is preparedbeforehand, and when original image data is input, the color conversiontable is referenced to produce output image data. Thus, in flesh colorcorrection, for example, a flesh color part of an image is made vivid onoutput.

In conventional image processing techniques mentioned above, however,when correction of certain chromaticity is made, an entire image may beaffected by the correction, causing an unsatisfactory result of imageadjustment. For instance, if it is attempted to make a pallid complexionof a person's image look ruddy, an entire image may become reddish.Namely, although a certain kind of image adjustment provides a desiredeffect on an image, it may cause an undesired side effect thereon.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide an imageprocessing apparatus which is capable of minimizing an adverse sideeffect in adjustment of an image data.

In accomplishing this object of the present invention and according toone aspect thereof, there is provided an image processing apparatus forinputting photographic image data containing dot-matrix pixels andconverting each pixel of the image data according to predeterminedcorrespondence relationship information, which comprises thecorrespondence relationship holding unit which holds a plurality ofcorrespondence relationship information of the image data in combinationwith applicable object information thereof, the correspondencerelationship judging unit which judges each correspondence relationshipfor converting each pixel of the image data according to thecorrespondence relationship and applicable object information which areheld by the correspondence relationship holding unit, and the image dataconverting unit which converts each pixel of the image data byreferencing each correspondence relationship judged by thecorrespondence relationship judging unit from the correspondencerelationship holding unit.

Where an image contains dot-matrix pixels and each pixel is indicated inimage data representation, each pixel of image data is convertedaccording to predetermined correspondence relationship information inthe above-mentioned arrangement of the present invention. In this imageprocessing, a plurality of correspondence relationship information forimage data conversion and applicable object information are held incombination. According to these information, a judgment is formed oneach correspondence relationship for conversion of each pixel of imagedata, and based on result of judgment, conversion of each pixel of imagedata is carried out.

More specifically, there are provided plural correspondencerelationships for image data conversion, and it is judged whichcorrespondence relationship is to be applied to each pixel. Then, imagedata conversion is performed by applying a proper correspondencerelationship to each pixel. For example, a correspondence relationshipfor making green color of tree leaves more vivid is applied to greencolor pixels indicating tree leaves, and a correspondence relationshipfor making a complexion ruddy is applied to flesh color pixels. In thismanner, each correspondence relationship is flexibly applied asrequired.

In accordance with the present invention, since correspondencerelationship information for image data conversion can be changed foreach pixel, it is possible to provide an image processing method whichallows execution of desired image processing without giving an adverseeffect to unspecified parts of an image.

It will be obvious to those skilled in the art that the image processingmethod in which plural correspondence relationship information can bechanged for each pixel is practicable not only in a substantialapparatus, but also in a system on its method. From this point of view,the present invention provides an image processing apparatus ofinputting photographic image data containing dot-matrix pixels andconverting each pixel of the image data according to predeterminedcorrespondence relationship information, in which a plurality ofcorrespondence relationship information for conversion of the image dataand applicable object information thereof are held in combination, eachcorrespondence relationship for converting each pixel of the image datais judged according to the correspondence relationship and applicableobject information thus held, and each pixel of the image data isconverted according to result of judgment.

The present invention for realizing image processing as mentioned abovemay be practiced in a variety of forms, i.e., it may be embodied inhardware, software or combination thereof in various arrangements thatare modifiable as required. As an example of practicing the presentinvention, it may be embodied in image processing software. In thiscase, according to the present invention, there is provided a softwarestorage medium for recording the image processing software. From thisviewpoint, the present invention provides a storage medium for recordingan image processing program for inputting photographic image datacontaining dot-matrix pixels on a computer and converting each pixel ofthe image data according to predetermined correspondence relationshipinformation, the image processing program comprising the steps of;holding a plurality of correspondence relationship information forconversion of the image data in combination with applicable objectinformation thereof, forming judgment on each correspondencerelationship for converting each pixel of the image data according tothe correspondence relationship and applicable object information thusheld, and converting each pixel of the image data according to result ofjudgment thus formed. In this case, the same functionalities as in theforegoing description can be provided.

The storage medium for recording the image processing program may be amagnetic storage medium, a magneto-optical storage medium or anysoftware storage medium to be developed in the future. Obviously, thepresent invention embodied in image processing software may take such areplicated form as a primary replicated product, secondary replicatedproduct, etc. In addition, the image processing software according tothe present invention may be supplied through use of a communicationline or in a form of contents written in a semiconductor chip.

Still more, there may be provided such an arrangement that some parts ofthe present invention are embodied in software while the other partsthereof are embodied in hardware. In a modified embodiment of thepresent invention, some parts thereof may be formed as software recordedon a storage medium to be read into hardware as required.

In image processing according to the present invention, photographicimage data is particularly suitable as object data to be processed.However, it is not necessarily required to use 100% photographic imagedata as object data, and semi-synthetic image data orcomposite-photographic image data produced from plural photographicimages may be used as object data.

As stated above, in accordance with the present invention, there areprovided a plurality of correspondence relationships in combination withapplicable object information thereof. Each correspondence relationshipis used to perform specific conversion of input image data, and for eachcorrespondence relationship, there is provided applicable objectinformation thereof. It is to be understood that such applicable objectinformation may take a variety of forms.

Another object of the present invention is to provide an imageprocessing method of using applicable object information.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship applicable toeach part of image is held along with information on each partapplicable to each correspondence relationship, and wherein, in thecorrespondence relationship judgment means, a part of imagecorresponding to each pixel is detected and each correspondencerelationship for conversion is judged through comparing the detectedpart of image with part information for each correspondencerelationship.

In the above-mentioned arrangement of the present invention, a uniquecorrespondence relationship is applicable to each part of image, andthere is provided part information applicable to each correspondencerelationship. Therefore, a part of image corresponding to each pixel isdetected, and the detected part of image is compared with partinformation for each correspondence relationship. Thus, if a match isfound, image data conversion is carried out using relevantcorrespondence relationship information.

More specifically, a unique correspondence relationship is provided foreach part, of image and it is applied to image data to be converted. Forinstance, a correspondence relationship for sky-blue color is applied toan upper part of image, while performing no conversion on a lower partof image. In this case, a part of image is not limited in its size andshape, and it may take any shape such as a rectangular shape, roundshape, free shape, etc. Furthermore, it is not necessarily required tospecify a part of image in a particular range. A part of image may bespecified with respect to a certain center point. In this case, a degreeof adjustment in image conversion may be decreased gradually as beingapart from the center point. Obviously, a certain correspondencerelationship may be applied to a plurality of parts of image.

As stated above, in accordance with the present invention, since eachcorrespondence relationship is changed with each part of image, it ispossible to carry out image processing with a particular locationspecified, thereby permitting easy operation.

Another object of the present invention is to provide an imageprocessing method of specifying an applicable object.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, a plurality of correspondence relationshipsare held along with information on chromaticity applicable to eachcorrespondence relationship, and wherein, in the correspondencerelationship judgment means, a value of chromaticity of each pixel isdetected and each correspondence relationship for conversion is judgedthrough comparing the detected value of chromaticity with information onchromaticity for each correspondence relationship.

In the above-mentioned arrangement of the present invention, a uniquecorrespondence relationship is applicable to each level of chromaticity.Therefore, a value of chromaticity of each pixel is detected, and thedetected value of chromaticity is compared with information onchromaticity for each correspondence relationship in judgment on eachcorrespondence relationship for conversion. In this case, chromaticityis used for the purpose of judgment on applicability of eachcorrespondence relationship, i.e., it is not necessarily required toadjust a level of chromaticity.

In accordance with the present invention, since an object can beselected using chromaticity, its embodiment is relatively easy even ifpixels to be processed are located at a plurality of parts of image.

In chromaticity judgment, input image data may be used intactly, or acolor space scheme is changed before judgment so that input image datais converted into data easy for judgment on chromaticity. Still more, itis not necessarily required to use a narrow definition of chromaticity.Chromaticity judgment may be made in a wider definition by using a levelof any predetermined recognizable characteristic of image data.

It is possible to adopt various units for setting correspondencerelationship information for image data conversion.

Another object of the present invention is to provide an imageprocessing apparatus of using a color conversion table.

According to the present invention, there is provided an imageprocessing method wherein, in the step of holding the correspondencerelationship information, a color conversion table is used for storinginformation on correspondence relationship between pre-convertedoriginal image data and post-converted image data.

In the above-mentioned arrangement of the present invention, the colorconversion table retains pre-converted image data and post-convertedimage data. By referencing the color conversion table usingpre-converted image data, it is possible to attain post-converted imagedata, i.e., correspondence relationship information is stored in a formof table.

Besides, it is possible to hold correspondence relationship informationin such forms as arithmetic expressions, arithmetic parameters, etc.However, in use of a table form, there is an advantage that operationscan be simplified since just referencing is required.

As stated above, in accordance with the present invention, sincecorrespondence relationship information is held in the color conversiontable, it is relatively easy to carry out image data conversion.

As to a plurality of correspondence relationships, it is not necessarilyrequired to provide different kinds of correspondence relationships.

Another object of the present invention is to provide an imageprocessing apparatus of attaining results of plural conversionssubstantially.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, a plurality of correspondence relationshipsare realized by changing a degree of correspondence relationshipapplicability.

In the above-mentioned arrangement of the present invention, a pluralitycorrespondence relationships are set up by changing a degree ofcorrespondence relationship applicability. For example, according to adegree of applicability, relational association can be made betweenimage data not applied to a certain level of correspondence relationshipand image data applied thereto. In this case, either linear ornon-linear relational association may be made.

Still more, in use of such a degree of applicability, just a singlecorrespondence relationship can set up a condition converted by applyingthe correspondence relationship and a condition not convertedsubstantially, i.e., it is possible to provide a plurality ofcorrespondence relationships. In this case, more correspondencerelationships can be realized by changing a degree of applicability asrequired.

As stated above, in accordance with the present invention, it ispossible to rearrange a single correspondence relationship as aplurality of correspondence relationships by changing a degree ofapplicability.

Another object of the present invention is to provide an imageprocessing method of holding a plurality of correspondence relationshipswhich may be predetermined or specified as required.

According to the present invention, there is provided an imageprocessing apparatus wherein, the correspondence relationshipinformation holding unit includes the correspondence relationshipspecifying unit in which a plurality of correspondence relationshipsapplicable to the image data are specified in succession as required.

In the above-mentioned arrangement of present invention, a plurality ofcorrespondence relationships to be applied to the image data arespecified in succession. There are two kinds of correspondencerelationship elements; a condition to be converted, and an object to beapplied. Either one or both of these kinds of elements may be taken. Inthis case, it is also possible to specify a part of each image andselect a conversion condition for the specified part of image. Stillmore, there maybe provided such an arrangement that a flesh coloradjustment item or a sky-blue color adjustment item is selectable as anoptional function of image processing.

As stated above, in accordance with the present invention, sinceapplicable correspondence relationships can be specified as required, itis possible to improve a degree of freedom in image processing.

Another object of the present invention is to provide an imageprocessing apparatus of holding each correspondence relationship foradjusting brightness.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingbrightness of the image data is held.

In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting brightness according to theimage data is held, and brightness of predetermined pixels is adjustedusing the correspondence relationship thus held. For example, on acertain part of image, brightness of pixels thereof is increased.

As stated above, in accordance with the present invention, it ispossible to adjust brightness on a part of image based on applicablecorrespondence relationship information.

Another object of the present invention is to provide an imageprocessing apparatus of holding each correspondence relationship foradjusting chromaticity.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingchromaticity according to the image data is held.

In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting chromaticity according to theimage data is held, and chromaticity of predetermined pixels is adjustedusing the correspondence relationship thus held. For example, on a partof image containing flesh color pixels, enhancement is made to provideruddy flesh color.

As stated above, in accordance with the present invention, it ispossible to adjust chromaticity on a part of image based on applicablecorrespondence relationship information.

Another object of the present invention is to provide an imageprocessing apparatus of adjusting color vividness.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingcolor vividness according to the image data is held.

In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting color vividness according tothe image data is held, and vividness of predetermined pixels isadjusted using the correspondence relationship thus held. For example,to enhance color of any subject on a surrounding background, coloradjustment is made so that the color of the subject becomes more vivid.

As stated above, in accordance with the present invention, it ispossible to adjust color vividness on a part of image based onapplicable correspondence relationship information.

If it is distinguished clearly whether a predetermined correspondencerelationship is to be applied or not, a conspicuous boundary may takeplace in an image. Therefore, another object of the present invention isto make such a boundary unrecognizable by providing an image processingapparatus for adjusting a level of correspondence relationshipgradually.

According to the present invention, there is provided an imageprocessing apparatus wherein, in the image data conversion unit, forpixels in a transition region between regions which are different interms of correspondence relationship, a level of correspondencerelationship is adjusted gradually.

In the above-mentioned arrangement of the present invention, atransition region is provided between regions which are different interms of correspondence relationship, and for pixels in the transitionregion, a level of correspondence relationship is adjusted gradually inimage data conversion, thereby eliminating a stepwise difference in aprocessed image.

As stated above, in accordance with the present invention, it ispossible to suppress occurrence of a stepwise difference on a boundarybetween regions to which different correspondence relationships areapplied respectively in an image.

A level of correspondence relationship to be adjusted gradually in atransition region is selectable as required, and either linear ornon-linear transition may be applied.

As mentioned in the foregoing description, correspondence relationshipinformation for image data conversion can be used for color adjustment.However, as to selection of an element color and a degree ofenhancement, it is required for a human operator to make a decision. Incolor adjustment processing of digital image data, it is difficult toidentify which part will cause conspicuous color deviation in advance.

For instance, in case of a color image of a building, color deviationwill not be recognized as long as an original color thereof is unknown.However, in case of a flesh color image of a person's face, an originalcolor thereof can be presumed in most cases. Therefore, adjustment offlesh color in an image is performed by the operator so that naturalflesh color is given.

In color adjustment, there is a problematic factor which is referred toas a memory color effect in psychology. When a lemon is photographed asa subject, for example, a color of a photographed lemon image looksslightly somber even if it meets a color of the actual lemon inmeasurement using a calorimeter. Agreement between the color of thelemon image and that of the actual lemon cannot be found unless aside-by-side comparison check is made. Due to a memory color effect inpsychology, a person memorizes that a color of a lemon is vivid ingeneral. However, an actual lemon does not have such a vivid color as ismemorized. Therefore, even if a lemon image is corrected to have itsactual color, it is not satisfactory in terms of visual sensation inpsychology. In case of flesh color, green color of tree leaves and bluecolor of sky, a memory color effect tends to occur as stated above.

When a person conducts adjustment of flesh color, for example, a degreeof flesh color enhancement is determined in an appropriate range thoughthe flesh color does not exactly match its memory color. If the fleshcolor is adjusted to exactly match its memory color, deviation willoccur in other colors.

Therefore, automatic color adjustment cannot be performed just bypredetermining correspondence relationship information. Even inautomatic color adjustment, it is rather difficult to attainsatisfactory result due to a memory color effect, i.e., a person mustreadjust result of automatic color adjustment.

It is another object of the present invention to provide a coloradjustment device, a color adjustment method and a storage medium forrecording a color adjustment control program for carrying out optimumcolor adjustment processing automatically while taking account of suchfactors as a memory color effect.

According to the present invention, there is provided a color adjustmentdevice for performing color separation of color image data for eachpredetermined element color and adjusting the color image data inenhancement to provide each desired color in result of color imageoutput delivered for each element color by such a device as an imageoutput device, etc., comprising; the chromaticity judging unit whichdetermines a value of chromaticity of each pixel according to the colorimage data, the object chromaticity pixel statistical calculation unitwhich performs statistical calculation on pixels having chromaticityvalues which have been determined to meet a predefined range ofchromaticity by the chromaticity judging unit, the color adjustmentdegree judging unit which determines a degree of color adjustment so asto eliminate a difference between a predetermined optimum value forpixels meeting the predefined range of chromaticity and a result valueattained in the statistical calculation and for regulating the degree ofcolor adjustment according to an occupancy ratio of pixels subjected tostatistical calculation to the total number of pixels, and the coloradjusting unit which carries out color adjustment of the image dataaccording to the regulated degree of color adjustment.

In the above-mentioned arrangement of the present invention, a value ofchromaticity of each pixel is determined according to the color imagedata. Since a value of chromaticity represents an absolute ratio inpsychophysical color specification which does not depend on brightness,important parts (objects) in an image can be identified according topossible ranges of chromaticity. For example, an object can bedistinguished by checking whether its chromaticity value is in apossible chromaticity range of flesh color or green color of treeleaves. Based on this principle of chromaticity, if determinedchromaticity values of pixels are in a predefined chromaticity range,statistical calculation is performed on these pixels. In the aboveexample, if determined chromaticity values of pixels are in the possiblechromaticity range of flesh color, they are subjected to statisticalcalculation. For statistical calculation, an average value, median, etc.may be taken. Statistical calculation on pixels having chromaticity in apredefined range with respect to all the pixels of an image is analogousto a situation that a person judges an average level of flesh color withrespect to flesh-color-like pixels.

Then, it is necessary to judge whether or not a particular color matchesits memory color. A degree of color adjustment is so determined as toeliminate a difference between a predetermined optimum value for pixelsmeeting a predefined range of chromaticity and a result value attainedin the statistical calculation. However, if a degree of color adjustmentthus determined is applied intactly, a flesh color part will be adjustedto have memory flesh color, for example, but non-allowable colordeviation will occur on other color parts. Therefore, the degree ofcolor adjustment thus determined is regulated according to an occupancyratio of pixels subjected to the statistical calculation to the totalnumber of pixels. In this manner, color adjustment is made properly sothat adjustment of flesh color will not cause an excessive change inother colors. Thus, the image data is color-adjusted according to theregulated degree of color adjustment.

As stated above, in accordance with the present invention, it ispossible to provide a color adjustment method which enables automaticcolor adjustment processing through chromaticity statistical calculationfor simulating human judgment.

It is to be understood that a method of performing statisticalcalculation on pixels having chromaticity in a predefined range andreflecting result of statistical calculation in a degree of coloradjustment is practicable in a substantial apparatus for itsimplementation. From this point of view, the present invention providesa color adjustment device for performing color separation of color imagedata for each predetermined element color and adjusting the color imagedata in enhancement to provide each desired color in result of colorimage output delivered for each element color by such a device as animage output device, etc., comprising the steps of; determining a valueof chromaticity of each pixel according to the color image data,performing statistical calculation on pixels having chromaticity valueswhich have been determined to meet a predefined range of chromaticity,determining a degree of color adjustment so as to eliminate a differencebetween a predetermined optimum value for pixels meeting the predefinedrange of chromaticity and a result value attained in the statisticalcalculation, regulating the degree of color adjustment according to anoccupancy ratio of pixels subjected to the statistical calculation tothe total number of pixels, and carrying out color adjustment of theimage data according to the regulated degree of color adjustment.

As an example of practicing the present invention, it may be embodied incolor adjustment control software for a color adjustment device. In thiscase, according to the present invention, there is provided a softwarestorage medium for recording the color adjustment control software. Fromthis viewpoint, the present invention provides a storage medium forrecording a color adjustment control program for performing colorseparation of color image data for each predetermined element color on acomputer and adjusting the color image data in enhancement to provideeach desired color in result of color image output delivered for eachelement color by such a device as an image output device, etc., thecolor adjustment control program comprising the steps of; determining avalue of chromaticity of each pixel according to the color image data,performing statistical calculation on pixels having chromaticity valueswhich have been determined to meet a predefined range of chromaticity,determining a degree of color adjustment so as to eliminate a differencebetween a predetermined optimum value for pixels meeting the predefinedrange of chromaticity and a result value attained in the statisticalcalculation, regulating the degree of color adjustment according to anoccupancy ratio of pixels subjected to the statistical calculation tothe total number of pixels, and carrying out color adjustment of theimage data according to the regulated degree of color adjustment. Inthis case, the same functionalities as in the foregoing description canbe provided.

In determining chromaticity of each pixel of color image data,chromaticity conversion is performed taking account of a color spacecoordinate scheme used for original color image data. In this case,chromaticity is used intactly if it is a direct element, or chromaticityis converted if it is not a direct element. For conversion, a conversiontable or a conversion expression may be used. In this case, it is notnecessarily required to attain an accurate value in result, i.e., anerror may be contained if its adverse effect is insignificant.

When determined chromaticity values of pixels are in a predefinedchromaticity range, statistical calculation is performed on thesepixels. In his case, it is not necessarily required to make analternative-choice judgment, i.e., it is not necessarily required todetermine whether the predefined chromaticity range is satisfied or not.Instead, statistical calculation may be made while changing a weightfactor.

A variety of statistical calculation approaches may be used inapplication.

Another object of the present invention is to provide a color adjustmentmethod which is advantageous in the amount of calculation.

According to the present invention, there is provided a color adjustmentdevice wherein: in object chromaticity pixel statistical calculationunit, an average value of pixels judged to be object pixels iscalculated for each element color of color image data; and in coloradjustment degree judging unit, an optimum value of each element coloris provided for color image data meeting the predefined range ofchromaticity, and a degree of color adjustment is determined accordingto the optimum value of each element color.

In the above-mentioned arrangement of the present invention, statisticalcalculation is performed for each element color of color image data todetermine an average value of pixels judged to be object pixels, i.e.,statistical calculation Is carried out using each element color of colorimage data instead of chromaticity statistical calculation. On the otherhand, an optimum value of each element color is provided for color imagedata meeting the predefined range of chromaticity. In judgment of adegree of color adjustment, comparison for each element color isperformed, and a difference attained is used as a degree of elementcolor adjustment.

In accordance with the present invention, it is judged from chromaticitywhether or not an object is applicable to statistical calculation. Sincestatistical calculation is performed for each element color of colorimage data, statistical result can be used easily in subsequentarithmetic operations.

Besides, it is possible to use median calculation, standard deviationcalculation, etc. though the amount of calculation increases instatistics.

A range of chromaticity to be subjected to statistical calculation canbe changed as required.

Another object of the present invention is to provide a color adjustmentdevice which is suitable for memory color adjustment.

According to the present invention, there is provided a color adjustmentdevice wherein, in the object chromaticity pixel statistical calculationunit, statistical calculation is performed on object chromaticity pixelsin a possible range of chromaticity in terms of memory color inpsychology.

In the above-mentioned arrangement of the present invention, statisticalcalculation is performed on object chromaticity pixels which meet apossible range of chromaticity in terms of memory color in psychology.In this case, it is not necessarily required to attain statisticalresult using just one possible range of chromaticity. It is possible tocarry out a plurality of statistical calculations respectively byspecifying plural possible chromaticity ranges.

In accordance with the present invention, since, calculation isperformed on pixels corresponding to memory color, a degree of coloradjustment is regulated according to the number of pixels, therebymaking it possible to form a judgment similar to that by a person.

In judgment of a degree of color adjustment, a degree of coloradjustment is so determined as to eliminate a difference between apredetermined optimum value for pixels meeting a predefined range ofchromaticity and a result value attained in the statistical calculation,and the degree of color adjustment is regulated according to anoccupancy ratio of pixels subjected to statistical calculation to thetotal number of pixels. In this case, it is not necessarily required tocarry out these two steps of processing. It is just required to providethe same result of processing. For example, if a difference between apredetermined optimum value for pixels meeting a predefined range ofchromaticity and a result value attained in statistical calculation isobtained in a step of statistical calculation, it may be reflected inresult of statistical calculation. If an occupancy ratio of pixelssubjected to statistical calculation to the total number of pixels isobtained in the step of statistical calculation, it may be reflected inresult of statistical calculation. Sill more, it is not necessarilyrequired to perform subtraction to attain a difference between apredetermined optimum value and a result value in statisticalcalculation. It is just required to include any proper calculationinherently.

A degree of color adjustment may also be determined in a variety offorms.

Another object of the present invention is to provide a color adjustmentdevice of determining a degree of color adjustment.

According to the present invention, there is provided a color adjustmentdevice wherein, in the color adjustment degree judging unit, atone curverepresenting input-output relationship information is used for changinga degree of element color enhancement, and a tone curve is formedaccording to the degree of color adjustment.

In the above-mentioned arrangement of the present invention, a tonecurve is formed according a degree of color adjustment, and a degree ofelement color enhancement is changed according to the tone curve thusformed so that input-output relationship information on each elementcolor is represented by the tone curve.

In accordance with the present invention, since a tone curve is used, itis easy to add adjusted conditions of other element colors from anoverall point of view. In this case, a memory color part is adjustedtoward an ideal color level and adjacent color regions are adjustedgradually in a continuous manner, thereby preventing occurrence ofpseudo-contouring due to tone jumping.

The above and other objects, features and advantages of the presentinvention will become more apparent from the following description ofembodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an image processing apparatus in apreferred embodiment of the present invention;

FIG. 2 is a block diagram showing concrete examples of hardware for theimage processing apparatus according to the present invention;

FIG. 3 is a schematic block diagram showing another example ofapplication of the image processing apparatus according to the presentinvention;

FIG. 4 is a schematic block diagram showing another example ofapplication of the image processing apparatus according to the presentinvention;

FIG. 5 is a flowchart of image processing in the image processingapparatus according to the present invention;

FIG. 6 is a diagram showing a state in which a processing object pixelis moved;

FIG. 7 is a diagram showing a display window for specifying a processingobject area and a processing item;

FIG. 8 is a diagram showing that two processing object areas areselected;

FIG. 9 shows a region reference table used for storing specified regionsand image processing objects;

FIG. 10 is a diagram showing a window for selecting a processing objectand a processing item;

FIG. 11 is a diagram showing rectangular regions specified withchromaticity “x-y”;

FIG. 12 is a diagram showing that center points are specified withchromaticity “x-y”;

FIG. 13 is a flowchart of applicability setting;

FIG. 14 is a graph showing a change in applicability when a center pointis specified with chromaticity “x-y”;

FIG. 15 is a graph showing a luminance distribution range in expansion;

FIG. 16 shows a conversion table for expanding a luminance distributionrange;

FIG. 17 is a graph showing a conversion relationship for expanding aluminance distribution range;

FIG. 18 is a graph showing a conceptual condition for increasingluminance by γ-curve correction;

FIG. 19 is a graph showing a conceptual condition for decreasingluminance by γ-curve correction;

FIG. 20 is a graph showing a correspondence relationship of luminancesubjected to γ-correction;

FIG. 21 is a diagram showing an unsharp mask comprising 5×5 pixels;

FIG. 22 is a graph showing a statistical condition of saturationdistribution;

FIG. 23 is a graph showing a relationship between minimum saturation ‘A’and saturation index ‘S’;

FIG. 24 is a diagram showing a display window which contains an area tobe processed in entire image processing and an area to be processed inpartial image processing;

FIG. 25 is a block diagram showing a color adjustment device in apreferred embodiment of the present invention;

FIG. 26 is a flowchart of color adjustment processing in the coloradjustment device according to the present invention;

FIGS. 27(a)-(c) are diagrammatic illustrations showing tone cures forconverting gradation data with predetermined degrees of enhancement;

FIG. 28 is a flowchart of color adjustment processing in a modifiedembodiment according to the present invention;

FIG. 29 is a diagram showing a window for selecting a color adjustmentobject for the color adjustment processing; and

FIG. 30 is a diagram illustrating an original photographic image to besubjected to the color adjustment processing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will now be described in detail by way of examplewith reference to the accompanying drawings.

FIG. 1 shows a block diagram of an image processing scheme using animage processing apparatus in a preferred embodiment of the presentinvention, and FIG. 2 presents a schematic block diagram illustrating aconcrete example of a hardware configuration.

Referring to FIG. 1, an image input device 10 supplies photographicimage data containing dot-matrix pixels of a photographic or pictureimage to an image processing apparatus 20, on which an object applicableto image processing and conditions thereof are specified and then imageprocessing is carried out for object pixels. Thereafter, the imageprocessing apparatus 20 delivers the image data thus processed to animage output device 30, on which the processed image is output in adot-matrix pixel form.

As shown in FIG. 2, a concrete example of the image input device 10 is ascanner 11, a digital still camera 12 or a video camcorder 14, aconcrete example of the image, processing apparatus 20 is a computersystem comprising a computer 21, a hard disk unit 22, a keyboard 23, amouse 27, a CD-ROM drive unit 24, a floppy disk drive unit, 25, a modem26, etc., and a concrete example of the image output device 30 is aprinter 31 or a display monitor 32. In the present preferred embodiment,photographic image data is most suitable as input data since objectpixels are specified for correcting deficiencies in an image and imageprocessing is performed according to a predetermined correspondencerelationship. The modem 26 can be connected to a public communicationline for downloading software and data from an external network throughthe public communication line.

In the present preferred embodiment, the scanner 11 or digital stillcamera 12 serving as the image input device 10 provides RGB (red, green,blue) gradation image data, the printer 31 serving as the image outputdevice 30 requires input of CMY (cyan, magenta, yellow) or CMYK (cyan,magenta, yellow, black) binary gradation data, and the display monitor32 requires input of RGB gradation data. In the computer 21, a printerdriver 21 b for the printer 31 and a display driver 21 c for the displaymonitor 32 are run under control of an operating system 21 a.

Furthermore, an image processing application 21 d is controlled by theoperating system 21 a for execution of processing thereof, and the imageprocessing application 21 d carries out predetermined image processingin conjunction with the printer driver 21 b or the display driver 21 cas required. Therefore, the computer 21 serving as the image processingapparatus 20 is used for receiving RGB gradation data and generatingoptimally image-processed RGB gradation data through image evaluationfor output onto the display monitor 32 by means of the display driver 21c. The computer 21 is also used for converting RGB gradation data intoCMY (or CMYK) binary data for output onto the printer 31 by means of theprinter driver 21 b.

As mentioned above, the computer system is arranged between the imageinput device and the image output device for carrying out imageevaluation and image processing in the present preferred embodiment. Thecomputer system is not always-required, however It is to be understoodthat the invention is applicable to a variety of image data processingapparatuses. As shown in FIG. 3, for example, there may be provided sucha system arrangement that an image processing apparatus for carrying outimage processing for each predetermined applicable object isincorporated in a digital still camera 12 a and converted image data isdisplayed on a display monitor 32 a or printed on a printer 31 a. Stillmore, as shown in FIG. 4, for a printer 31 b capable of receiving imagedata and printing images without using a computer system, there may beprovided such an arrangement that image data is input from a scanner 11b, a digital still camera 12 b or modem 26 b to the printer 31 b, inwhich image processing is carried out through judgment on whether or noteach pixel belongs to each applicable object.

In the embodiments mentioned above, image evaluation and imageprocessing are implemented by means of an image processing programflowcharted in FIG. 5, which is run on the computer 21, for example.Referring to FIG. 5, the image processing program is executed throughthe following steps: At the first step S100, an applicable object isspecified for each image processing. At steps S110 to S140, apredetermined image processing operation is performed by moving anobject pixel for each dot-matrix pixel as shown in FIG. 6. Morespecifically, at step S110, it is judged whether each object pixel isapplicable to image processing conditions specified at S100. At stepS120, image data is converted according to the result of judgment. Inthis conversion, image processing is accomplished in effect. At stepsS130 and S140, each object pixel is moved as required. The followingdescribes this image processing flow in further detail.

Depending on image processing conditions, it is determined which pixelsare to be subjected to image processing. In the present preferredembodiment, a part of image to be processed is specified as shown inFIGS. 7 and 8. Referring to FIG. 7, there are provided an operation area41 along a top side of a window 40 for window frame operation, a displayarea 42 at a center part of the window 40, and a processing menu area 43at a bottom side of the window 40 for image processing menu items.

An image to be processed is displayed on the display area 42, arectangular region is specified using the mouse 27, and an imageprocessing menu item is selected on the processing menu area 43 usingthe mouse 27. The processing menu area 43 contains executable imageprocessing items including ‘CONTRAST’ adjustment, ‘BRIGHTNESS’adjustment, ‘SHARPNESS’ adjustment and ‘SATURATION’ adjustment items,each of which is provided with arrows for specifying a degree ofadjustment. For instance, clicking an up-arrow of ‘CONTRAST’ adjustmentitem with the mouse 27 specifies image processing for increasingcontrast, and clicking a down-arrow thereof specifies image processingfor decreasing contrast.

For each specified region of image, a plurality of image processingitems may be selected. When ‘LIST DISPLAY/EDIT’ item on the processingmenu area 43 is clicked, information on a selected image processing itemis displayed as shown in a region reference table exemplified in FIG. 9,i.e., an upper left corner coordinate point and a lower right cornercoordinate point of each specified region, a specified kind of imageprocessing, and a specified level of adjustment are displayed. A levelvalue indicates a degree of image processing adjustment, which can beincreased/decreased by clicking an up/down-arrow of each imageprocessing item on the processing menu area 43 as many times asrequired. A kind-of-processing code indicates a particular kind of imageprocessing as follows: In the example shown in FIG. 9, the CONTRASTadjustment item is indicated as ‘1’, the BRIGHTNESS adjustment item isindicated as ‘2’, the SHARPNESS adjustment item is indicated as ‘3’, andthe SATURATION adjustment item is indicated as ‘4’. The kinds of imageprocessing and levels of adjustment will be described in more detailtogether with description of image processing conditions. For removing aimage processing item from the region reference table, it is justrequired to highlight the image processing item by clicking with themouse and press a DELETE key on the keyboard 23. Besides, the contentsof the region reference table can be edited in other manners similar tothose in ordinary application software.

In the present preferred embodiment, since the image processingapplication 21 d is executed at an application level of the operatingsystem 21 a, an image be processed can be displayed in the window 40 anda region of interest can be specified in it. However, in an instancewhere the same image processing functionality is implemented at a driverlevel of the operating system 21 a, e.g., in the printer driver 21 b, animage to be processed may not be displayed in a window.

FIG. 10 shows an example in which an object of image processing isspecified regardless of whether windowing display is provided or not.When the printer driver 21 b is made active, an window 50 is selectableas an option. While a part of image to be processed is specified in theexamples shown in FIG. 7 to 9, pixels to be processed are selected byspecifying image chromaticity this example. The following describeschromaticity used in the invention.

As an example of representation in chromaticity, “x-y” chromaticitycalculation is performed below. Assuming that RGB gradation datacontaining pixels to be processed in RGB color scheme is (R, G, B), thefollowing expressions are given.r=R/(R+G+B)   (1)g=G/(R+G+B)   (2)

Then, in XYZ color scheme, the following correspondence relationship isset up between chromaticity coordinates “x” and “y”.x=(1.1302+1.6387r+0.6215g)/(6.7846−3.0157r−0.3857g)   (3)y=(0.0601+0.9399r+4.5306g)/(6.7846−3.0157r−0.3857g)   (4)

Since a value of chromaticity represents an absolute ratio inpsychophysical color specification which does not depend on brightness,it is possible to identify an object to which the pixels of interestbelong according to chromaticity thereof.

For example, in an object image of a person, it is necessary to extractpixels in a flesh color part. A value of chromaticity of flesh color isin a range expressed below.0.35<x <0.40   (5)0.33<y<0.36   (6)

Hence, if a determined chromaticity value of each pixel is within therange indicated above, it can be assumed that the pixel of interestcorresponds to a point of skin of a person's image in most cases.Similarly, as to such object images as blue sky and green leaves oftrees, chromaticity is also applicable in judgment. Regardless ofbrightness, pixels of blue sky and green leaves of trees can beidentified approximately. FIG. 11 shows a distribution relationship of“x-y” chromaticity, in which there are indicated applicable rectangularregions corresponding to flesh color, sky-blue color and green color ofleaves. As shown in FIG. 10, there may be provided such an arrangementthat a range value of “x-y” chromaticity distribution can be specifieddirectly for individual definition.

FIG. 12 shows a technique for specifying a center point in determinationof each applicable region. In case of flesh color, a center point ofits, applicable region is found as expressed below using Equations (5)and (6).x=0.375   (7)y=0.345   (8)

Then, with reference to this center point, it is checked whether a valueof chromaticity is within a range of a predetermined radius or not. Ifthis condition is satisfied, it is judged that each pixel of interestbelongs to a particular applicable region.

In the window 50 selectable using an optional function of the printerdriver 21 b, there are provided adjustment items which indicatepractical examples corresponding to predefined chromaticity ranges to bespecified for an object so that a human operator can easily adjust acolor of blue sky or green leaves of trees as desired. In addition,there are provided arrow buttons which are used for increasing ordecreasing chromaticity of pixels of interest.

In the example shown in FIG. 10, a range of pixels is specified withchromaticity and it is determined whether the chromaticity is to beenhanced or not in image processing. However, it is not always necessaryto let chromaticity-based pixel selection be consistent with imageprocessing to be performed. For instance, in image processing, it isallowed to increase brightness of flesh color pixels or contrast ofgreen color pixels. Namely, chromaticity is used only as an indexindicating whether pixels of interest are to be processed or not. Inthis example, chromaticity is used just for the purpose of pixelselection in image processing.

As described above, step S100 at which an applicable object is specifiedfor each image processing comprises the unit for specifying acorrespondence relationship. However, it is to be understood that stepS100 is not limited to this function and may be modified to provide theunit for specifying a part of image or a value of chromaticity asrequired. There may also be provided such an arrangement that otherfactors including brightness and sharpness are specified. As shown inFIG. 9, the region reference table for indicating a specified region andholding its image processing in a correspondence relationship is storedin a memory area of the computer system in the present preferredembodiment. It will be obvious to those skilled in the art that theregion reference table is used to provide the unit for holding acorrespondence relationship in combination with a color conversion tablefor image processing (to be described later).

At step S110 and the subsequent steps, each object pixel is moved and itis judged whether each object pixel is applicable to a specifiedoperation of image processing as mentioned above. For checking whethereach object pixel is to be subjected to image processing or not, theregion reference table shown in FIG. 9 is used. A coordinate location ofeach object pixel is compared with upper left corner and lower rightcorner coordinates indicated in each field of the region referencetable. If the coordinate location of each object pixel meets acoordinate value indicated in any field, it is judged that the objectpixel of interest is applicable to image processing. If it does not meetany coordinate value indicated in any field, it is judged that theobject pixel of interest is not applicable to image processing. In casethat chromaticity is used for judgment as shown in FIG. 10, object pixelchromaticity is calculated using Equations (1) to (4), and it is judgedwhether a calculated value is within a chromaticity range selected usingan optional function of the printer driver 21 b.

Whether an object is applicable to image processing or not is judgedaccording to a part of image or chromaticity as mentioned above.However, if an alternative-choice judgment is made to determine whethera specific range is satisfied or not, any visual stepwise difference isprone to occur in an adjacent region. For instance, if brightness of acertain rectangular region is increased while brightness of an immediateoutside part thereof is not increased, a stepwise difference inbrightness occurs on a boundary therebetween, resulting in therectangular region being distinguishable clearly.

To circumvent this adverse effect, the present preferred embodiment usesa parameter of image processing applicability k. Referring to FIG. 13,if an object is judged to belong to a specified region at step Sill,applicability k is set to ‘1’ at step S112. For a transition region on acircumference of the specified region, applicability k is changed in arange of ‘0’ to ‘1’. More specifically, if the object is judged to bewithin the transition region at step S113, applicability k is adjustedtoward ‘1’ at step S114 in case the object is near the specified regionor it is adjusted toward ‘0’ in case the object is apart from thespecified region. If the object is judged not to belong to the specifiedregion nor the transition region, applicability k is set to ‘0’ at stepS115. While a center point of chromaticity is specified as exemplifiedin FIG. 12, there may be provided such an arrangement as shown in FIG.14. Referring to FIG. 14, applicability k is set to ‘1’ for an imageprocessing object corresponding to radius ‘r0’, and under condition thata transition region is provided in a range of radius ‘r0’ to ‘r1’,applicability k is gradually adjusted toward ‘0’ in a circumferentialregion.

At step S110, an applicable correspondence relationship is judged, andat steps S111 to S115, applicability k is determined as mentioned above.According to the present preferred embodiment, there is provided theunit for judging a correspondence relationship using software processingincluding such procedural steps as steps S110 to S115 mentioned aboveand hardware for implementation thereof. While applicability k isdetermined as a result of judgment in each image processing in thepresent preferred embodiment, it will be obvious to those skilled in theart that an alternative-choice judgment is also feasible in application.

At step S120, image processing is carried out for each object pixelaccording to applicability k. The following describes more specificconditions of image processing in practicing the present invention.

Contrast indicates a width of luminance in an entire image, and inmostcases of contrast adjustment, it is desired to increase a width ofcontrast. Referring to FIG. 15, there is shown a histogram ofstatistical luminance distribution of pixels of a certain image. In caseof a solid-line curve of narrow distribution, a difference in luminanceamong bright and dark pixels is relatively small. In case of adot-dashed-line curve, of broad distribution, a difference in luminanceamong bright and dark pixels is relatively large, which signifies that awidth of contrast is increased. Referring to FIG. 17, there is shown agraph indicating a luminance conversion operation for enhancingcontrast. Assuming that the following equation holds for a relationshipbetween original unconverted luminance y and converted luminance Y,Y=ay+b   (9)

Through conversion under condition “a>1”, a difference between a maximumvalue of luminance ‘ymax’ and a minimum value of luminance ‘ymin’ isincreased, resulting in a broad distribution of luminance as shown inFIG. 15. In this case, it is preferred to determine slope ‘a’ and offset‘b’ according to luminance distribution. For example, the followingequations are given:a=255/(ymax−ymin)   (10)b=−a·ymin or 255−a·ymax   (11)

Under the conditions indicated above, a certain narrow luminancedistribution can be expanded to a reproducible extent. However, if it isexpanded to an extreme end of a reproducible range, highlights may beblown out to white or shadows may be plugged up to black. To preventthis, a non-expand able margin corresponding to a luminance value ofapprox. ‘5’ is provided at each of the upper and lower limits of thereproducible range. Resultantly, conversion parameters are expressed asfollows:a=245/(ymax−ymin)   (12)b=5−a·ymin or 250−a·ymax   (13)

In this case, conversion is not performed in ranges where “y<5” and“y>250”.

In image data conversion, it is not required to perform calculation eachtime. If a luminance value range is ‘0’ to ‘255’, a result of conversionis predetermined for each luminance value and a conversion table isprepared as shown in FIG. 16. In this case, the conversion table isusable just for luminance. In an instance where image data containsdirect elements of luminance, the conversion table can be used.Contrarily, in an instance where image data contains only indirectelements of luminance, the conversion table cannot be used. In mostcomputer systems, red, green and blue elements of image data areindicated in gradation of brightness, i.e., color gradation data (R, G,B) is used. Since such gradation data (R, G, B) does not provide directvalues of luminance, it is necessary to perform conversion to Luv colorspace scheme for determining luminance. This method is, however,disadvantageous since a large amount of calculation is required.Therefore, the following conversion expression is used, which iscommonly adopted in television signal processing for directlydetermining luminance from RGB data:y=0.30R+0.59G+0.11B   (14)

On a principle that linear conversion can be made between gradation dataand luminance y as indicated above, Equation (9) is applicable to arelationship between original unconverted gradation data (R0, G0, B0)and converted gradation data (R1, G1, B1). Then, the followingexpressions are given:R1=aR0+b   (15)G1=aG0+b   (16)B1=aB0+b   (17)

Consequently, the conversion table shown in FIG. 16 is applicable togradation data conversion.

The following describes an image processing technique for adjustingbrightness. As in the above-mentioned case of contrast adjustment, ahistogram of brightness distribution is assumed. Referring to FIG. 18, asolid-line curve indicates a luminance distribution which has its peakinclined toward a dark level. In this case, the peak of entiredistribution is shifted toward a bright side as indicated by abroken-line curve. Referring to FIG. 19, a solid-line curve indicates aluminance distribution which has its peak inclined toward a bright side.In this case, the peak of entire distribution is shifted toward a darkside as indicated by a broken-line curve. In these cases, linearluminance conversion as shown in FIG. 17 is not performed, but γ-curveluminance conversion is performed as shown in FIG. 20.

In γ-curve correction, entire brightness is increased when “γ<1”, and itis decreased when “γ>1”. A degree of this correction can be adjustedgradually by clicking an up-arrow or down-arrow of the BRIGHTNESSadjustment item on the processing menu area 43 shown in FIG. 7 as manytime as required at step S100.

As in contrast adjustment, it is also possible to set up a value of γautomatically. As a result of our various experiments, it has been foundthat the following approach is advantageous: In a luminancedistribution, median ‘ymed’ is predetermined. If it is less than ‘85’,an image of interest is judged to be too dark and γ correction is madeaccording to a γ value indicated below.γ=ymed/85   (18)or,γ=(ymed/85)**(1/2)   (19)

Note, however, that even if “γ<0.7”, a value of γ is set to 0.7forcedly. Unless this kind of limit is provided, a night-scene imageappears to be of daytime. If an image is brightened excessively, itbecomes whitish entirely, resulting in low contrast. In this case, it ispreferred to perform such a processing operation as enhancement ofsaturation in combination.

If median ‘ymed’ is larger than ‘128’, an image of interest is judged tobe too bright and γ correction is made according to a γ value indicatedbelow.γ=ymed/128   (20)or,γ=(ymed/128)**(1/2)   (21)

In this case, a limit is also provided so that a value of γ is set to1.3 forcedly even if “γ>1.3” for the purpose of preventing the image ofinterest from becoming too dark.

For this γ correction, it is preferred to provide such a conversiontable as shown in FIG. 16.

In edge enhancement processing for adjusting sharpness of an image, withrespect to original non-enhanced luminance Y of each pixel, enhancedluminance Y′ is calculated as expressed below.Y′=Y+Eenhance*(Y−Yunsharp)   (22)

where ‘Eenhance’ indicates a degree of edge enhancement, and ‘Yunsharp’indicates unsharp-mask processing for each pixel of image data. Thefollowing describes the unsharp-mask processing: Referring to FIG. 21,there is shown an example of an unsharp mask 60 comprising 5×5 pixels.The unsharp mask 60 is used in summation in such a manner that a centervalue of ‘100’ is assigned as a weight to a processing object pixel‘Y(x,y)’ in dot-matrix image data and a weight corresponding to a valuein each array box of the unsharp mask 60 is assigned to eachcircumferential pixel thereof. In use of the unsharp mask 60, summationis performed based on the following expression: $\begin{matrix}{{{Yunsharp}\left( {x,y} \right)} = {\left( {1/396} \right){\sum\limits_{ij}\left( {{Mij} \times {Y\left( {{x + i},{Y + j}} \right)}} \right)}}} & (23)\end{matrix}$

In Equation (23), ‘396’ indicates a total value of weighting factors.For an unsharp mask having a different size, a total of values in arrayboxes thereof is indicated. ‘Mij’ represents a weighting factorindicated in each array box of an unsharp mask, and ‘Y(x,y)’ representseach pixel of image data. In the unsharp mask 60, ‘i’ and ‘j’ indicatecoordinates on horizontal and vertical axes thereof.

Edge enhancement calculation based on Equation (22) provides thefollowing functional meaning: ‘Yunsharp(x,y)’ is the result of additionin which a weight assigned to each circumferential pixel is reduced withrespect to a pixel of interest, and accordingly an image is unsharpenedthrough processing. Such an unsharpening operation is functionallyequivalent to low-pass filtering. Therefore, “Y(x,y)−Yunsharp(x,y)”signifies that a low-frequency component is subtracted from each oforiginal components, i.e., it is functionally equivalent to high-passfiltering. If a high-frequency component subjected to high-passfiltering is multiplied by a degree of edge enhancement ‘Eenhance’ and aresult value of this multiplication is added to “Y(x,y)”, it means thata high-frequency component is increased in proportion to the degree ofedge enhancement ‘Eenhance’. Thus, edge enhancement is accomplished.Since edge enhancement is required only for an edge part of an image, itis possible to reduce the amount of processing substantially byperforming only in case that there is a large difference betweenadjacent pixels of image data.

In this case, the degree of edge enhancement ‘Eenhance’ can be adjustedby clicking an up-arrow or down-arrow of the SHARPNESS adjustment itemon the processing menu area 43 shown in FIG. 7 as many times as requiredat step S100. Still more, it is possible to set up the degree of edgeenhancement ‘Eenhance’ automatically.

At an edge part of an image, a difference in gradation data increasesbetween adjacent pixels. This difference represents a gradient ofluminance, which is referred to as a degree of edging. In an image, adegree of variation in luminance can be calculated by determining eachof horizontal-direction and vertical-direction vector components.Although an object pixel in an image containing dot-matrix pixels isadjacent to eight pixels, a degree of variation is determined only withrespect to adjacent pixels in horizontal and vertical directions for thepurpose of simplifying calculation. Summation is performed on lengthvalues of respective vectors to represent a degree of edging ‘g’ for theobject pixel of interest, and a sum value of edging degree is divided bythe number of pixels to attain an average value. Assuming that thenumber of pixels is indicated as ‘E(I)pix’, a degree of sharpness ‘SL’of an object image can be calculated as expressed below.SL=Σ|g|/E(I)pix   (24)

In this case, as a value of ‘SL’ decreases, the degree of sharpnessbecomes lower (blurring). As a value of ‘SL’ increases, the degree ofsharpness becomes higher (clearer imaging).

Since sharpness of an image depends on a visual sensation of anindividual person, a degree of sharpness ‘SL’ is determined similarlyusing image data which has optimum sharpness attained experimentally. Avalue thus determined is set as an ideal level of sharpness ‘SLopt’, anda degree of edge enhancement ‘Eenhance’ is determined as expressedbelow.Eenhance=ks·(SLopt−SL)**(1/2)   (25)

where coefficient ‘ks’ varies with a size of image. In case that imagedata contains ‘height’ dots and ‘width’ dots in vertical and horizontaldirections respectively, coefficient ‘ks’ can be determined as indicatedbelow.ks=min (height, width)/A   (26)

where ‘min (height, width)’ indicates the number of ‘height’ dots or thenumber of ‘width’ dots, whichever is smaller, and ‘A’ is a constantvalue of ‘768’.

It is to be understood that the above value has been attained fromexperimental results and may be altered as required. Basically, as animage size increases, it is advisable to increase the degree of edgeenhancement.

In the above-mentioned fashion, edge enhancement processing can becarried out in manual or automatic setting.

The following describes an image processing technique for adjustingsaturation. In case of saturation adjustment using a saturationenhancement parameter ‘Sratio’, the parameter can be changed as requiredfor such image data that has saturation parameters. For attaining asaturation value from gradation data containing RGB component valuesonly, it is primarily necessary to perform conversion to a color spacescheme in which saturation values are used as direct component values.However, in this processing, RGB image data is converted into Luv-spaceimage data, and then after saturation enhancement, it is re-convertedinto RGB image data again, resulting in an increase in the amount ofcalculation. Therefore, RGB gradation data is directly subjected tosaturation enhancement.

In RGB color space scheme in which components are represented by huecomponent values having approximately equivalent relationships,condition “R=G=B” signifies gray without saturation. It is thereforeconsidered that a minimum value of each component indicates just adecrease in saturation without affecting hue of each pixel. Based onthis principle, saturation can be enhanced by subtracting a minimumvalue of each component from all the component values and increasing aresultant difference value of subtraction.

If a component value of blue (B) is minimum in RGB gradation datacontaining red, green and blue components (R, G, B), conversion isperformed as shown below using the saturation enhancement parameter‘Sratio’.R′=B+(R−B)×Sratio   (27)G′=B+(G−B)×Sratio   (28)B′=B   (29)

Thus, since there is no need to perform conversion and re-conversionbetween RGB color space scheme and Lub space scheme, a time required forprocessing can be reduced substantially. While the present preferredembodiment uses a technique of subtracting a minimum component valuefrom another component value as to a non-saturated component, adifferent conversion equation may be used for subtracting a value of anon-saturated component. In case that just a minimum value is subtractedas in Equations (27) to (29), the amount of processing is relativelysmall since multiplication or division is not involved.

In saturation adjustment using Equations (27) to (29), satisfactoryconversion can be performed but there is a tendency that an imagebecomes brighter entirely due to an increase in luminance whensaturation is enhanced. Therefore, conversion is performed for adifference value attained by subtracting a value corresponding toluminance from each component value. Assuming that saturationenhancement is expressed as shown below,R′=R+ΔR   (30)G′=G+ΔG   (31)B′=B+ΔB   (32)

Each of the above operands ΔR, ΔG and ΔB can be determined according toa difference value with respect to luminance as shown below.ΔR=(R−Y)×Sratio   (33)ΔG=(G−Y)×Sratio   (34)ΔB=(B−Y)×Sratio   (35)

Hence, conversion can be carried out as expressed below.R′=R+(R−Y)×Sratio   (36)G′=G+(G−Y)×Sratio   (37)B′=B+(B−Y)×Sratio   (38)

As to retention of luminance, the following emulations are applicable:Y′=Y+ΔY   (39)ΔY=0.30 ΔR+0.59 ΔG+1.11 ΔB =Sratio {(0.30 R+0.59 G+0.11 B)−Y}=0   (40)

In case of input of gray (R=G=B), condition “luminance Y=R=G=B” is setup, resulting in condition “operand ΔR=ΔG=ΔB=0”. Thus, no color is givenin case of non-saturation. In use of Equations (36) to (38), luminancecan be retained and an image does not become brighter entirely even whensaturation is enhanced.

In this case, the saturation enhancement parameter ‘Sratio’ can beadjusted by clicking an up-arrow or down-arrow of the SATURATIONadjustment item on the processing menu area 43 shown in FIG. 7 as manytimes as required at step 100. Still more, it is possible to set up thesaturation enhancement parameter “Sratio” automatically.

A value of pixel saturation can be determined in a simplified manner.For this purpose, a substitute value ‘X’ of saturation is calculated asshown below.X=|G+B−2×R|  (41)

Essentially, a value of saturation becomes ‘0’ under condition “R=G=B”,and it becomes maximum when a single color of red, green and blue isgiven or two colors thereof are mixed at a predetermined ratio. Based onthis nature, it is possible to represent a value of saturation directly.Yet, using the simple Equation (41), a maximum value of saturation isprovided for a single color of red or a mixture color of cyan producedby mixing green and blue, and a value of saturation is indicated as ‘0’when each component is uniform. For a single color of green or blue,approximately half a level of maximum is provided. It is obvious thatsubstitution into the following equations is also possible.X′=|R+B−2×G|  (42)X″=G+R−2×B   (43)

In a histogram distribution as to the substitute value ‘X’ ofsaturation, saturation levels are distributed in a range of minimum ‘0’to maximum ‘511’ as schemed in FIG. 22. Then, according to thestatistical saturation distribution shown in FIG. 22, a saturation index‘S’ of an image of interest is determined. In the saturationdistribution, a range of upper ‘16%’ is defined, and a minimumsaturation value ‘A’ in the defined range is taken to representsaturation of the image of interest.

If A<92, thenS=−A×(10/92)+50   (44)

If 92≦A<184, thenS=−A×(10/46)+60   (45)

If 184≦A<230, thenS=−A×(10/23)+100   (46)

If 230≦A, thenS=0   (47)

In this manner, the saturation index ‘S’ is determined. Referring toFIG. 23, there is shown a relationship between minimum saturation ‘A’and saturation index ‘S’. As shown in this figure, the saturation index‘S’ increases in a range of maximum ‘50’ to minimum ‘0’ as thesaturation ‘A’ decreases, and it decreases as the saturation ‘A’increases. For conversion from the saturation index ‘S’ to thesaturation enhancement parameter ‘Sratio’, the following equation isapplicable:Sratio=(S+100)/100   (48)

In this case, when the saturation index ‘S’ is ‘0’, the saturationenhancement parameter ‘Sratio’ becomes ‘1’ not to allow saturationenhancement.

The following describes a technique for enhancing chromaticity whilespecifying a range of pixels according to chromaticity as shown in FIG.10. In principle, an approach for increasing luminance of pixels inchromaticity enhancement is employed. Therefore, a γ-correction tonecurve as shown in FIG. 20 is used. According to a degree of chromaticityenhancement, a value of γ can be adjusted, and it is also possible toset up a γ value automatically, which will be described later.

In the present preferred embodiment, there are provided such imageprocessing techniques as mentioned hereinabove. Since a correspondencerelationship applicable to an object pixel is judged at step S110 andapplicability k is determined at steps S111 to S115, image dataconversion is performed at step S120 according to result of judgment.

As mentioned above, on the processing menu area 43 shown in FIG. 7, akind of image processing is selected and a level of enhancement isspecified. Therefore, at step S100, an image processing conversion tablebased on respective enhance levels for image processing is prepared andstored into a predetermined memory area of the computer system. Then, atstep S120, the image processing conversion table is referenced to carryout conversion as described below.

Under condition that components of pre-converted RGB gradation data are(Rpre, Gpre, Bpre), components of post-converted RGB gradation dataattained through reference to a predetermined conversion table are(Rpost, Gpost, Bpost), and components of final image data are (Rfinl,Gfinl, Bfinl), conversion is performed as expressed below.Rfinl=k·Rpost+(1−k)·Rpre   (49)Gfinl=k·Gpost+(1−k)·Gpre   (50)Bfinl=k·Bpost+(1−k)·Bpre   (51)

In image processing based on these equations, a gradual increase inweight assignment is made in a transition region where applicability kvaries from ‘0’ to ‘1’, resulting in elimination of a stepwisedifference.

Equations (49) to (51) are used in succession for all the imageprocessing conversion tables to which the object pixel is applicable. Ifapplicability k is ‘0’, it is not necessary to perform image dataconversion. As to Equations (49) to (51), each of red, green and bluecomponents is subjected to calculation, but just one or two of thesecomponents may be calculated in some cases. Still more, in such asituation where applicability k is not used, each component (Rpost,Gpost, Bpost) of RGB gradation data attainable just by changing a kindof conversion table may be used intactly.

In the foregoing description, an applicable object is specified for eachimage processing, i.e., only a particular region in view of an entireimage is subjected to image processing. It is however possible toperform additional image processing operation on a particular regionwhile carrying out a certain image processing operation in the entireimage.

Under condition that each component (Rtotal, Gtotal, Btotal) can beattained as a result of image processing conversion for the entire imageand each component (Rpart, Gpart, Bpart) can be attained as a result ofimage processing conversion for any particular region, weighted additionis performed using applicability k′ similar to the applicability asshown below.Rfinl=k′·Rpart+(1−k′)·Rtotal   (52)Gfinl=k′·Gpart+(1−k′)·Gtotal   (53)Bfinl=k′·Bpart+(1−k′)·Btotal   (54)

Referring to FIG. 24, there is shown a schematic example in which animage of a slightly back-lighted person in a scene is selected toincrease its brightness while entire color vividness is enhanced. Aresult of conversion on a region of the entire image (Rtotal, Gtotal,Btotal), hatched by frontward-sloping diagonal lines, is laid over aresult of conversion on a region of the person's image (Rpart, Gpart,Bpart), hatched by backward-sloping diagonal lines, according toapplicability k′. In this case, on the region of the person's image,applicability k′ is set to a maximum level of 0.5, and on a transitionregion at a circumference thereof, applicability k′ is adjusted in arange of 0<k′<0.5. Still more, if applicability k′ is set to a maximumlevel of 1.0 on the region of the person's image and applicability k′ isadjusted in a range of 0<k′<1.0 on the transition region at thecircumference thereof, conversion for the entire image can be excludedfrom the region of the person's image. In the same manner, amultiplicity of image processing results can be applied.

Thereafter, at step S130, the object pixel is moved, and at step S140,it is checked whether all the object pixels have been subjected toprocessing. If processing is completed for all the object pixels, theimage processing sequence concerned comes to an end.

Having described the present preferred embodiment as related totechniques for automatically setting up a degree of enhancement inrespective image processing operations, there may also be provided suchan arrangement that image data is sampled uniformly for statisticalcalculation at a step before movement of an object pixel as mentionedabove and a degree of enhancement is set up automatically according toresult of statistical calculation while a conversion table is created.

In an instance where a range of chromaticity is selected using anoptional function of the printer driver 21 b, an enhancement level canbe set up automatically in such a manner as mentioned above, therebymaking it possible to improve operability. The following describesoperations in the preferred embodiment arranged as stated above.

It is assumed that a photographic image is read in using the scanner 11and printed out using the printer 31. First, under condition that theoperating system 21 a is run on the computer 21, the image processingapplication 21 d is launched to let the scanner 11 read in thephotographic image. When the photographic image thus read in is takeninto the image processing application 21 d under control of theoperating system 21 a, the image processing application 21 d carries outimage processing operations as flowcharted in FIG. 5.

At step S100, the read-in photographic image is presented on the displayarea 42 of the window 40 as shown in FIG. 7 so that an applicable objectcan be specified. In this state, a human operator specifies arectangular region of sky-blue part using the mouse 27 and clicks theup-arrow of the SATURATION adjustment item on the processing menu area43 as many times as desired for saturation enhancement. Still more, theoperator specifies a rectangular region of person's image part at thecenter and clicks the up-arrow of the BRIGHTNESS adjustment item on theprocessing menu area 43 as many times as desired for brightnessenhancement. Namely, while specifying an image processing operation ofsaturation enhancement on a rectangular region corresponding to a skypart of background, the operator specifies an image processing operationof brightness enhancement on a rectangular region corresponding to aperson's image part.

When the window 40 is closed on completion of specifications, aconversion table is created according to the specifications. That is, akind of image processing is determined with reference to akind-of-processing code indicated on the region reference table shown inFIG. 9, and a degree of enhancement is judged according to alevel-of-adjustment value indicated thereon for conversion tablecreation. In this example, a conversion table for saturation enhancementprocessing and a conversion table for brightness enhancement processingare created.

Thereafter, an object pixel applicable to processing is set at aninitial position, and at step S110, it is judged whether or not acoordinate location of the object pixel is included in the regionreference table shown in FIG. 9. In this case, a transition region isalso taken into account, and applicability k is attained. If the objectpixel is located in a hatched region as shown in FIG. 8, applicability kis set to ‘1’. Then, at step S120, with reference to a conversion tablefor applicable image processing, the image data is converted accordingto Equations (49) to (51). In an example shown in FIG. 8, therectangular region specifying the background part and the rectangularregion specifying the person's image part are overlaid partially, and anoverlaid part is subjected to image data conversion through two stepsusing Equations (49) to (51). In a transition region, image dataconversion is also performed to eliminate a stepwise difference along anon-specified circumferential part.

The above-mentioned operations are repeated while moving each objectpixel applicable to processing at step S130 until it is judge at stepS140 that all the pixels have been processed. Thus, saturation isenhanced on the sky-blue part to provide a clear blue sky image, whilethe person's part is brightened to provide such a blight image asphotographed with flash light even if the person's image is back-lightedin scene. It is to be understood that the blue-sky part is notbrightened due to enhancement of brightness of the person's part andsaturation of the person's part is not increased due to enhancement ofsaturation of the blue-sky part.

Thereafter, image data thus processed is displayed on the displaymonitor 32 through the display driver 21 c, and if the image thusdisplayed is satisfactory, it is printed out on the printer 31 throughthe printer driver 21 b. More specifically, the printer driver 21 breceives RGB gradation image data which has been subjected to imageprocessing as specified for each specified region, performs resolutionconversion as predetermined, and carries out rasterization according toa print head region of the printer 31. Then, the image data thusrasterized is subjected to RGB-to-CMYK color conversion, and thereafter,CMYK gradation image data is converted into binary image data for outputonto the printer 31.

In an instance where the printer driver 21 b is made active by a requestfor print processing from a certain application unlike theabove-mentioned image processing through the image processingapplication 21 d, an image to be read in may not be displayed on thedisplay area 42 of the window 40. In this case, the printer driver 21 bcan present such an option selection window as shown in FIG. 10. On thiswindow, the operator specifies a desired image processing item to attaina clear flesh color part of a person's image or a vivid green part oftree leaves, for example, while observing an original photograph or thelike. This processing operation corresponds to step S100 in FIG. 5 atwhich an applicable object is specified.

After an optional processing item is selected, the printer driver 21 bcreates a conversion table internally and judges chromaticity of eachpixel of input image data. Thus, it is checked whether chromaticity ofeach pixel is applicable to the selected optional processing item. If itis applicable, image processing conversion is carried out usingEquations (49) to (51) with reference to the conversion table, and imagedata thus converted is further converted into CMYK image data which canbe printed out onto the printer 31.

In such a manner as mentioned above, flesh color pixels of a person'simage and green color pixels of tree images in an original scene areenhanced through adjustment, resulting in vivid imaging on printout.

Thus, on the computer 21 serving as the nucleus of image processing, aregion applicable to image processing is specified at step S100, anobject pixel is moved for judging whether or not it belongs to thespecified region at steps S110 to S140, and then a specified imageprocessing operation is carried out if the object pixel is judged tobelong to the specified region. Therefore, adjustment of image data in acertain region does not have an adverse effect on image data in anotherregion, making it possible to realize satisfactory adjustment in anentire image with ease.

While chromaticity enhancement is made in a range of pixels specifiedaccording to chromaticity in the example mentioned above, it is alsopracticable to set up a degree of enhancement automatically. Thefollowing describes details of a color adjustment device implemented forthe purpose of automatic setting of a degree of enhancement.

Referring to FIG. 25, there is shown a block diagram of a coloradjustment device in a preferred embodiment of the present invention. Aconcrete example of a hardware configuration of this system is similarto that illustrated in FIG. 2.

In FIG. 25, an image input device 70 supplies photographic color imagedata containing dot-matrix pixels to a color adjustment device 80, onwhich optimum color adjustment processing for the color image data iscarried out. Then, the color adjustment device 80 deliverscolor-adjusted image data to an image output device 90, on which acolor-adjusted image is output in a dot-matrix pixel form. In thissequence, the color-adjusted image data delivered by the coloradjustment device 80 is produced through judging an object and color ofimage according to chromaticity of each pixel and determining aprinciple and degree of optimum color adjustment in terms of colorcorrection of an entire image. For this purpose, the color adjustmentdevice 80 comprises the unit for judging chromaticity, the unit forstatistical calculation of object chromaticity pixels, the unit forjudging a degree of color adjustment, and the unit for adjusting color.

To be more specific, judgment on an object and color adjustment thereofare carried out by a color adjustment processing program as flowchartedin FIG. 26, which is run on the computer 21. Note that a flowchartexemplified in FIG. 26 is for color adjustment processing to provideclear flesh color.

In the color adjustment processing, statistical calculation is performedon flesh-color-like pixels according to chromaticity of each pixel. Asshown in FIG. 6, each object pixel is moved for statistical calculationon all the pixels.

First, at step S210, chromaticity “x-y” of each pixel is calculated. Asin the case of the example in the foregoing description, flesh color isidentified if the following expressions are satisfied.0.35<x<0.40   (5)0.33<y<0.36   (6)

At step S220, it is judged whether or not chromaticity “x-y” convertedaccording to each pixel of RGB gradation data is in a predefined fleshcolor range. If it is in the flesh color range, statistical calculationis performed on each pixel of color image data at step S230. Thisstatistical calculation signifies simple addition of RGB gradation datavalues. The number of pixels is also counted to determine an averagevalue for pixels judged to have flesh color, which will be described indetail later.

Thereafter, regardless of whether or not each object pixel is judged tohave flesh color, each object pixel is moved at step S240. Thus, theabove-mentioned sequence is repeated until it is judged at step S250that processing for all the pixels is completed. On completion ofprocessing for all the pixels, step S260 is performed to dividestatistical result data by the number of pixels for determining anaverage value (Rs.ave, Gs.ave, Bs.ave).

Software processing for “x-y” chromaticity calculation at step S210 andhardware for execution thereof provide the unit for judgingchromaticity. It is judged at step S220 whether or not chromaticity“x-y” is in a predetermined object range, and if the predeterminedobject range is satisfied, statistical calculation is performed on colorimage data at step S230. Then, at steps S240 and S250, each object pixelis moved until all the pixels are taken. At step S260, statisticalresult is divided by the number of pixels to determine an average value.These software processing operations and hardware for execution thereofprovide the unit for statistical calculation of object chromaticitypixels.

As to pixels having preferable flesh color, an ideal value (Rs.ideal,Gs.ideal, Bs.ideal) is predefined. In terms of memory color inpsychology, each ideal value is different from result of actualmeasurement. In an example of flesh color, a person tends to have anoptical illusion that slightly deviated flesh color is true rather thanflesh color conforming to actual measurement result. This opticalillusion is based on a stereotyped recognition of flesh color onphotographs and pictures, i.e., it is referred to as a Memory coloreffect in psychology. In the present invention, an ideal value ispredefined in consideration of such a memory color effect so that coloradjustment is made to eliminate a difference from the ideal value.Therefore, the ideal value may be in a wide target range expectedwithout being biased to actual color.

Regarding flesh color pixels, a difference between an average value(Rs.ave, Gs.ave, Bs.ave) of RGB gradation data and an ideal value(Rs.ideal, Gs.ideal, Bs.ideal) predefined for preferable flesh colorrepresents a degree of deviation in color image data fundamentally.

However, it is not preferred to apply the difference as a degree ofcolor adjustment intactly. For instance, if the same degree of coloradjustment is applied to all the pixels, a flesh color part may becomesatisfactory but colors of pixels on any parts other than the fleshcolor part may be affected significantly.

In the present preferred embodiment, therefore, a ratio of the number offlesh color pixels to the total number of pixels (flesh color ratio) isdetermined at step S270 for regulating a degree of color adjustment.Degrees of adjustment of primary colors ΔR, ΔG and ΔB are expressed asshown below.ΔR=ks(Rs.ideal−Rs.ave)   (55)ΔG=ks(Gs.ideal−Gs.ave)   (56)ΔB=ks(Bs.ideal−Bs.ave)   (57)

Based on these equations, a value of flesh color ratio ‘ks’ is attainedas indicated below.ks=(Number of flesh color pixels/Total number of pixels)

A degree of color adjustment thus attained is not applied intactly tocolor image data adjustment. In the present preferred embodiment, a tonecurve is prepared using the degree of color adjustment at step S280.FIG. 27 is a diagrammatic illustration showing tone curves prepared inthe present embodiment.

A tone curve represents an input-output relationship where RGB gradationdata is converted with a degree of enhancement regulated. In an exampleof 256 gradations ranging from levels ‘0’ to ‘255’, a spline curve isdrawn with respect to three identified output value points correspondingto gradation level ‘0’, gradation level ‘255’ and a certain mediumgradation level therebetween. Assuming that medium gradation level ‘64’is taken and output values are ‘0’, ‘64’ and ‘255’, there is acoincidence in an input-output relationship even if input values are‘0’, ‘64’ and ‘255’, resulting in a tone curve being straightened.However, if output value ‘64’ is not provided for input value ‘64’, agentle curve as shown in FIG. 27 is drawn to set up an input-outputrelationship. In the present preferred embodiment, a control pointcorresponding to the average value (Rs.ave, Gs.ave, Bs.ave) of RGBgradation data is used as a medium gradation level, and respectivedegrees of color adjustment ΔR, ΔG and ΔB are reflected in formation ofa tone curve. In this fashion, the control point is changed so that eachideal value (Rs.ideal, Gs.ideal, Bs.ideal) is met when the flesh colorratio ‘ks’ is ‘1’.

At step S290, element colors of color image data are converted for allthe pixels again using a tone curve thus attained to accomplish coloradjustment of the color image data.

Software processing at step S270 where color adjustment is madeaccording to a ratio of the number of object pixels to the total numberof pixels while determining a difference between a statistical resultvalue and an ideal value, software processing at step S280 where a tonecurve is formed according to a determined degree of color adjustment,and hardware for execution thereof provide the unit for judging a degreeof color adjustment. Software processing at step S290 where color imagedata is converted and hardware for execution thereof provide the unitfor adjusting color.

Having described the present preferred embodiment as related to fleshcolor adjustment for the purpose of simplicity in explanation, it is tobe understood that color adjustment is not limited to flesh color. Inconsideration of a memory color effect in psychology, it is oftendesired to attain more vivid green color of tree leaves and clearer bluecolor of sky in addition to clearer flesh color through color adjustmentprocessing. Referring to FIG. 28, there is shown a modified embodimentin which an object of adjustment is selectable.

In the example shown in FIG. 28, an object of color adjustment isselected first at step S305. In use of the computer 21, a window shownin FIG. 29 is presented on the display monitor 32 so that a humanoperator can select an object of color adjustment. The windowexemplified in FIG. 29 is provided with a flesh color adjustment itemfor clearer flesh color, a green color adjustment item for more vividgreen color of tree leaves, and a blue color adjustment item for clearersky blue, each of which has a check box for allowing individualselection. In this example, duplicate selection is also permitted. Whenthe operator turns on a desired check box and click the ‘OK’ button, aflag for each object thus specified is set up to start a loop processingof steps S310 to S350.

In the loop processing, while moving an object pixel, statisticalcalculation is performed on each pixel through determining chromaticityas in the foregoing description. At step S310, object pixel chromaticity“x-y” is determined using Equations (1) to (4). Then, at step S315, aflesh color adjustment flag which has been set up at step S305 isreferenced to judge whether or not the operator has selected the fleshcolor adjustment item. If it has been selected, statistical processingfor flesh color pixels is performed. This statistical processing iscarried out in the same manner as at steps S220 and S230 in the previousexample. If chromaticity “x-y” determined at step S310 is in apredefined possible chromaticity range corresponding to flesh color,statistical calculation is performed for each element color of RGBgradation data.

At step S325, a green color adjustment flag is referenced to judgedwhether or not the operator has selected the green color adjustment itemin the same manner as for flesh color adjustment. If the green coloradjustment item has been selected, object pixel chromaticity “x-y” ischecked to judge whether or not it is in a predefined possiblechromaticity range corresponding to green color of tree leaves. If it isin the corresponding predefined chromaticity range, statisticalcalculation is performed at step S330. This statistical calculation iscarried out in an area different from that subjected to flesh colorstatistical calculation.

Then, at step S335, the blue color adjustment item is checked to form ajudgment in the same manner, and at step S340, statistical calculationis performed on another area.

At step S345, each object pixel is moved, and the above-mentionedsequence is repeated until it is judged at step S350 that processing forall the pixels is completed. In this modified embodiment, a plurality ofobjects may be selected for color adjustment. Even in such a situation,statistical calculation is performed at steps S315 to S340 ifchromaticity of each object pixel is in a predefined chromaticity range.Therefore, these processing operations provide the unit for statisticalcalculation of object chromaticity pixels.

At steps S355 to S365 after completion of chromaticity statisticalcalculation on all the pixels, a degree of adjustment for each color iscalculated according to result of statistical calculation. Unlike theprevious example, a processing operation for determining an averagevalue from statistical calculation result is performed simultaneouslywith calculation of a degree of each color adjustment in this modifiedembodiment, and it is possible to modify relevant calculation proceduresas required. As described in the previous example, each adjustment offlesh color, green color and blue color s carried out in the samemanner. That is, an average value is calculated according to statisticalcalculation result, a difference between it and an ideal valuepredefined for preferable color is determined, and multiplication by aflesh color ratio, green color ratio or blue color ratio is performedfor regulating a degree of each color adjustment.

Upon completion of step S365, there are provided three kinds of degreesof color adjustment since degrees of flesh color adjustment, green coloradjustment and blue color adjustment have been determined throughrespective statistical calculations. Therefore, in the modifiedembodiment, addition is performed to reflect results of thesestatistical calculations inclusively. More specifically, in a situationof wider application of processing objects, degrees of color adjustmentfor respective processing objects ΔR, ΔG and ΔB are determined asexpressed below. $\begin{matrix}{{\Delta\quad R} = {\sum\limits_{i}{{ki}\quad\left( {{{Ri}.{ideal}}\text{-}{{Ri}.{ave}}} \right)}}} & (58) \\{{\Delta\quad G} = {\sum\limits_{i}{{ki}\quad\left( {{{Gi}.{ideal}}\text{-}{{Gi}.{ave}}} \right)}}} & (59) \\{{\Delta\quad B} = {\sum\limits_{i}{{ki}\quad\left( {{{Bi}.{ideal}}\text{-}{{Bi}.{ave}}} \right)}}} & (60) \\{{{where}\quad{\sum\limits_{i}{ki}}}<=1} & (61)\end{matrix}$

i=1: flesh color

i=2: green color

i=3: blue color

In this case, it is assumed that no duplicate counting is made.

At step S370, a tone curve is formed according to each of degrees ofcolor adjustment ΔR, ΔG and ΔG thus determined, as shown in FIG. 27. Inthis case, control points are indicated by Σ ki·Ri.ave, Σ ki·Gi.ave, Σki·Bi.ave. Software processing operations at steps S355 to S370 providethe unit for judging degree of color adjustment. After formation of eachtone curve, color image data is adjusted at step S375.

The above-mentioned color adjustment device may also be implemented as aprinter driver. Inmost cases, a printer driver is not capable oftemporarily storing data in an output process after processing of inputdata. Hence, there is a certain limitation in functionality for changingprocessing conditions according to each region divided as desired.However, by setting up degrees of color adjustment for a plurality ofelements as shown in Equations (58) to (60), it is possible to carry outeffective color adjustment even for the printer driver having such afunctional limitation.

The following describes operations of a preferred embodiment arrangedwith a printer driver.

As in the previous exemplary embodiment, it is assumed that aphotographic color image shown in FIG. 30 is read in using the scanner11 and printed out using the printer 31. First, under condition that theoperating system 21 a is run on the computer 21, the color adjustmentapplication 21 d is launched to let the scanner 11 read in thephotographic image. When the photographic image thus read in is takeninto the color adjustment application 21 d under control of theoperating system 21 a, an object pixel applicable to processing is setat an initial position. Then, at step S210, chromaticity “x-y” of eachpixel is calculated using Equations (1) to (4). At step S220, it isjudged whether or not each of values ‘x’ and ‘y’ is in a predefinedflesh color chromaticity range. If it is in the flesh color chromaticityrange, statistical calculation is performed on each pixel of color imagedata for each element color at step S230. In the photographic imageshown in FIG. 30, pixels of person's hands, legs or face can be judgedto have flesh color. In this example, statistical calculation isperformed on a few percent of all the pixels as flesh color pixels.Then, at step S240, each object pixel is moved. Thus, theabove-mentioned sequence is repeated until it is judged at step S250that processing for all the pixels is completed. After completion ofprocessing for all the pixels, statistical result data is divided by thenumber of flesh color pixels to determine an average value at step S260.At step S270, a difference between an ideal value of flesh color and theaverage value of flesh color pixels is determined, and it is multipliedby a flesh color ratio that represents a ratio of the number of fleshcolor pixels to the total number of pixels. At step S280, a tone curveis formed accordingly. Then, at step S290, based on the tone curve, eachelement color of color image data is converted for color adjustment.Since a degree of color adjustment is set at a moderate level withrespect to the ideal value of flesh color in consideration of the ratioof the number of flesh color pixels to the total number of pixels,preferable color can be attained through proper color adjustment.

In the example shown in FIG. 30, a difference between an average valueof flesh color pixels attained in statistical calculation and an idealvalue of flesh color is multiplied by a flesh color ratio indicating afew percent value for regulating a degree of color adjustment. Accordingto the regulated degree of color adjustment, a tone curve is formed foraccomplishing color adjustment.

Still more, if a flesh color adjustment item, green color adjustmentitem and blue color adjustment item are selected in adjustment objectselection as exemplified before, chromaticity “x-y” is calculated forall the pixels at step S310. Then, at steps S315 to S340, individualstatistical calculation is performed for each adjustment object. In theexample shown in FIG. 30, on a flesh color part of a person's image, agreen color part of tree leaves and a sky-blue part of a background,each chromaticity “x-y” is applicable to each object range forstatistical calculation.

After completion of processing for all the pixels, a degree of coloradjustment for each object is determined in consideration of anoccupancy ratio of object pixels at steps S355 to S365. At step S370, atone curve is formed through regulating each degree of color adjustmentthus determined. Then, at step S375, color adjustment is carried out forall the pixels of color image data. In this manner, flesh coloradjustment for attaining clearer flesh color, green color adjustment forattaining more vivid green color of tree leaves, and blue coloradjustment for attaining clearer sky blue in background are carried outthrough regulation according to the occupancy ratio of object pixels ofeach color.

After color adjustment thus accomplished, a color image is displayed onthe display monitor 32 through the display driver 21 c, and then if thecolor image thus displayed is satisfactory, it is printed out onto theprinter 31 through the printer driver 21 b. More specifically, theprinter driver 21 b receives RGB gradation image data which has beensubjected to color adjustment, performs resolution conversion aspredetermined, and carries out rasterization according to a print headregion of the printer 31. Then, the image data thus rasterized issubjected to RGB-to-CMYK color conversion, and thereafter, CMYKgradation image data is converted into binary image data for output ontothe printer 31.

Through the above-mentioned processing, the photographic color imageread in using the scanner 11 is automatically subjected to optimum coloradjustment. Thereafter, it is displayed on the display monitor 32, andthen printed out onto the printer 31.

As set forth hereinabove, on the computer serving as the nucleus ofcolor adjustment, chromaticity “x-y” of each pixel is calculated at stepS210, and statistical calculation is performed at steps S220 to S230 ifa value of chromaticity thus calculated is in a chromaticity rangepredefined for each color. After completion of statistical calculationon all the pixels, an average value is determined at step S260, and adegree of each color adjustment is calculated while taking account of anoccupancy ratio of object pixels of each color at step S270. In thisfashion, accurate statistical calculation is performed on color pixelsto be adjusted independently of brightness, and a degree of each coloradjustment is regulated by taking account of the number of pixels ofeach color in terms of occupancy ratio, thereby making it possible tocarry out optimum color adjustment processing without giving an adverseeffect on colors of pixels surrounding object pixels.

The invention may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. The presentembodiments are therefore to be considered in all respects asillustrative and not restrictive, the scope of the invention beingindicated by the appended claims rather by the foregoing description andall changes which come within the meaning and range of equivalency ofthe claims are therefore intended to be embraced therein.

1. An image processing apparatus for inputting image data of aphotographed image containing a plurality of pixels and for convertingthe image data of each of the pixels, the apparatus comprising: a firstholding unit which holds a plurality of correspondence relationshipinformation for a conversion of the image data; a second holding unitwhich holds relationship information related to an applicable object ofan image processing, correlating it to process content informationrepresenting details of the image processing; an applicable objectjudgment unit which judges, with reference to the relationshipinformation, whether a pixel corresponding to the input image data isthe applicable object of the image processing; a correspondencerelationship specification unit which, in a case where it is judged thatthe pixel is the applicable object of the image processing, withreference to the process content information correlated to thereferenced relationship information, specifies the correspondencerelationship which actualizes the contents of the image processing fromamong the plurality of correspondence relationships; and an image dataconverting unit which, in the image data of a pixel on which thespecification of the correspondence relationship is performed, makes theconversion on the image data of the pixel, with reference to thespecified correspondence relationship, and in the image data pixels in atransition region between regions being different in terms ofcorrespondence relationship, makes the conversion on the image data ofthe pixel in such a way that a level of correspondence relationship isgradually adjusted in image data conversion.
 2. An image processingapparatus as claimed in claim 1, wherein the second holding unit is aunit which holds a plurality of combinations of the relationshipinformation and the process content information; the applicable objectjudgment unit is a unit which carries out the judgment, with referenceto the plurality of each relationship information; and thecorrespondence relationship specification unit is a unit which, withreference to the process content information correlated to thereferenced plurality of each relationship information, specifies thecorrespondence relationship which actualizes each of the contents of aplurality of kinds of image processing.
 3. An image processing apparatusas claimed in claim 1, wherein the second holding unit holds, as therelationship information, position relationship information related tothe image processing applicable object position, and the applicableobject judgment unit is a unit which, including a unit which detects aposition in the photographed image of a pixel corresponding to the inputimage data, cross references the detected image position with theposition relationship information, and judges whether the pixelcorresponding to the image data is the image processing applicableobject.
 4. An image processing apparatus as claimed claim 1, wherein thesecond holding unit holds, as the relationship information, colorrelationship information related to an image processing applicableobject color, and the applicable object judgment unit is a unit which,including a unit which detects a chromaticity of the input image data,cross references the detected chromaticity with the color relationshipinformation, and judges whether a pixel corresponding to the image datais the image processing applicable object.
 5. An image processingapparatus as claimed in claim 1, wherein the first holding unit holdsthe correspondence relationship information in a tone curve form.
 6. Animage processing apparatus as claimed in claim 1, wherein the firstholding unit holds the correspondence relationship information in a formof a color conversion table which represents a correspondencerelationship between pre-converted image data and post-converted imagedata.
 7. An image processing apparatus as claimed in claim 1, whereinthe first holding unit holds the plurality of correspondencerelationship information as another correspondence relationshipgenerated by changing one correspondence relationship and its degree ofcorrespondence relationship applicability.
 8. An image processingapparatus as claimed in claim 1, wherein the second holding unit is aunit which holds the color relationship information, input by anoperation of a user, and the process content information.
 9. An imageprocessing apparatus as claimed in claim 1, wherein the first holdingunit holds, as the correspondence relationship, a relationship whichchanges a brightness based on the image data.
 10. An image processingapparatus as claimed in claim 1, wherein the first holding unit holds,as the correspondence relationship, a relationship which converts imagedata, and the image data conversion unit is a unit which converts thechromaticity of the image data based on the one specified correspondencerelationship.
 11. An image processing apparatus as claimed in claim 1,wherein the first holding unit holds, as the correspondencerelationship, a relationship which changes a color vividness based onthe image data.
 12. An image processing method for inputting image dataof a photographed image containing a plurality of pixels and convertingthe image data of each of the pixels, comprising: as well as holding aplurality of correspondence relationship information for a conversion ofthe image data, holding relationship information related to anapplicable object of an image processing, correlating it to processcontent information representing details of the image processing;judging, with reference to the relationship information, whether a pixelcorresponding to the input image data is the applicable object of theimage processing; in a case where it is judged that the pixel is theapplicable object of the image processing, with reference to the processcontent information correlated to the referenced relationshipinformation, specifying the correspondence relationship which actualizesthe contents of the image processing from among the plurality ofcorrespondence relationships; and in the image data of a pixel on whichthe specification of the correspondence relationship is performed,making the conversion on the image data of the pixel, with reference tothe specified correspondence relationship, and in the image data pixelsin a transition region between regions being different in terms ofcorrespondence relationship, making the conversion on the image data ofthe pixel in such a way that a level of correspondence relationship isgradually adjusted in image data conversion.
 13. A storage mediumreadable by a computer, which serves to store a program for convertingimage data of a photographed image containing a plurality of pixels, theprogram of instructions executable by the computer to perform functionsfor: judging, with reference to the image processing applicable objectrelationship information, whether a pixel corresponding to the inputimage data is the applicable object of the image processing; in a casewhere it is judged that the pixel is the applicable object of the imageprocessing, with reference to the process content information which,representing details of the image processing, is correlated to thereferenced relationship information, specifying the correspondencerelationship which actualizes the contents of the image processing fromamong the plurality of correspondence relationships; and in the imagedata of a pixel on which the specification of the correspondencerelationship is performed, making the conversion on the image data ofthe pixel, with reference to the specified correspondence relationship,and in the image data pixels in a transition region between regionsbeing different in terms of correspondence relationship, making theconversion on the image data of the pixel in such a way that a level ofcorrespondence relationship is gradually adjusted in image dataconversion.